Skip to content

fix(factory): suppress concurrent duplicate dispatches#44

Merged
khaliqgant merged 1 commit into
mainfrom
fix/live-dispatch-inflight-lock
Jun 24, 2026
Merged

fix(factory): suppress concurrent duplicate dispatches#44
khaliqgant merged 1 commit into
mainfrom
fix/live-dispatch-inflight-lock

Conversation

@khaliqgant

@khaliqgant khaliqgant commented Jun 24, 2026

Copy link
Copy Markdown
Member

Summary

  • Add an in-process dispatch promise lock keyed by Linear issue so concurrent live events for the same issue share the active dispatch instead of starting a second spawn sequence.
  • Keep retry/failure behavior unchanged after the active dispatch promise settles.

Why

During the real factory E2E run, duplicate live Linear events for the same issue could overlap while implementer/reviewer spawn was still in progress. The second dispatch attempted to spawn an already-created agent, hit agent already exists, abandoned inFlight, and prevented the PR babysitter handoff from retaining state.

Verification

  • npm run build
  • npx vitest run src/orchestrator/factory.test.ts -t "does not double-dispatch the same issue from concurrent live events"
  • Real provider proof: Linear AR-337 dispatched ar-337-impl-default + ar-337-review; GitHub PR docs(ar-337): add clean dispatch proof for 20260624T073141Z #43 opened; factory spawned ar-337-babysit; unclear Linear AR-339 posted Slack writeback at /slack/channels/C0BAC2KSTGV/messages/factory-c0bac2kstgv-cp49rh.json.

Summary by cubic

Prevents double-dispatch for the same Linear issue by sharing the in-flight dispatch across concurrent events, including canonical and alias paths. This avoids duplicate agent spawns and state loss.

  • Bug Fixes
    • Added #dispatchInFlight promise lock keyed by issue key + mode (dry-run/live) + phase (dispatch/escalation).
    • Concurrent events for the same key (across mirror paths) await the same dispatch; no second spawn.
    • Lock clears when the promise settles; retry/failure behavior unchanged.
    • Counts suppressed duplicates via dispatchDuplicateSuppressed.

Written for commit c010b20. Summary will update on new commits.

Review in cubic

@coderabbitai

coderabbitai Bot commented Jun 24, 2026

Copy link
Copy Markdown

Review Change Stack

📝 Walkthrough

Walkthrough

FactoryLoop in src/orchestrator/factory.ts gains a private #dispatchInFlight map that tracks one Promise<DispatchResult> per Linear issue key. The existing dispatch() method is split into a public wrapper that deduplicates concurrent calls and a new private #dispatchUnlocked() that holds the original dispatch logic.

Changes

Concurrent dispatch deduplication

Layer / File(s) Summary
In-flight map and dispatch wrapper
src/orchestrator/factory.ts
Adds private #dispatchInFlight: Map<string, Promise<DispatchResult>> field. Refactors dispatch() to return the existing in-flight promise when the same issue key is already being processed, otherwise delegates to a new #dispatchUnlocked() method that holds the prior implementation, registering and cleaning up the promise entry via a finally block.

Estimated code review effort

🎯 2 (Simple) | ⏱️ ~5 minutes

Possibly related PRs

  • AgentWorkforce/factory#36: Also modifies FactoryLoop.dispatch() in src/orchestrator/factory.ts, touching the same method now refactored into #dispatchUnlocked().

Poem

🐇 A rabbit hops in, then hops in again—
But the warren says "Wait! You're already in flight!"
One promise per burrow, no duplicates here,
The map keeps its keys with finally clear.
No doubling of work when the issues are tight! ✨

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Title check ✅ Passed The title clearly matches the main change: preventing concurrent duplicate dispatches in the factory.
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
Description check ✅ Passed The description matches the code change: it adds an in-process lock to prevent concurrent duplicate dispatches for the same Linear issue.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch fix/live-dispatch-inflight-lock

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request introduces a mechanism to suppress duplicate concurrent dispatches for the same issue by tracking in-flight dispatch promises in src/orchestrator/factory.ts. Feedback on the changes points out a potential issue where concurrent dispatches with different dryRun options could share the same in-flight promise, potentially suppressing a real dispatch with a dry-run dispatch. It is recommended to incorporate the dryRun option into the cache key to prevent this behavior.

Important

The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.

Comment on lines 1291 to +1308
async dispatch(decision: TriageDecision, opts: { dryRun?: boolean } = {}): Promise<DispatchResult> {
const key = issueKey(decision.issue)
const inFlight = this.#dispatchInFlight.get(key)
if (inFlight) {
this.#increment('dispatchDuplicateSuppressed')
return inFlight
}

const dispatched = this.#dispatchUnlocked(decision, opts)
this.#dispatchInFlight.set(key, dispatched)
try {
return await dispatched
} finally {
if (this.#dispatchInFlight.get(key) === dispatched) {
this.#dispatchInFlight.delete(key)
}
}
}

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

high

If dispatch is called concurrently with different dryRun options (for example, one real run and one dry run), they will share the same in-flight promise because the cache key only uses the issue key. This can lead to a real dispatch being suppressed by a dry-run dispatch, preventing agents from actually being spawned.

To resolve this, calculate the effective dryRun value and include it in the #dispatchInFlight map key.

Suggested change
async dispatch(decision: TriageDecision, opts: { dryRun?: boolean } = {}): Promise<DispatchResult> {
const key = issueKey(decision.issue)
const inFlight = this.#dispatchInFlight.get(key)
if (inFlight) {
this.#increment('dispatchDuplicateSuppressed')
return inFlight
}
const dispatched = this.#dispatchUnlocked(decision, opts)
this.#dispatchInFlight.set(key, dispatched)
try {
return await dispatched
} finally {
if (this.#dispatchInFlight.get(key) === dispatched) {
this.#dispatchInFlight.delete(key)
}
}
}
async dispatch(decision: TriageDecision, opts: { dryRun?: boolean } = {}): Promise<DispatchResult> {
const dryRun = opts.dryRun ?? this.#config.dryRun
const key = issueKey(decision.issue) + ':' + dryRun
const inFlight = this.#dispatchInFlight.get(key)
if (inFlight) {
this.#increment('dispatchDuplicateSuppressed')
return inFlight
}
const dispatched = this.#dispatchUnlocked(decision, opts)
this.#dispatchInFlight.set(key, dispatched)
try {
return await dispatched
} finally {
if (this.#dispatchInFlight.get(key) === dispatched) {
this.#dispatchInFlight.delete(key)
}
}
}

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 04d1441a6a

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread src/orchestrator/factory.ts Outdated
}

async dispatch(decision: TriageDecision, opts: { dryRun?: boolean } = {}): Promise<DispatchResult> {
const key = issueKey(decision.issue)

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Use Linear issue key for the dispatch lock

This lock is meant to suppress duplicate dispatches for the same Linear issue, but issueKey(decision.issue) includes the UUID and mount path (key:uuid:path), while the persisted dispatch guard is keyed only by issue.key. When concurrent detections for the same Linear key arrive through different issue paths/UUIDs (a scenario already covered by the repo's duplicate-path test case), both calls miss this in-process map and can still race past #dispatchBlockReason before either records inFlight, leaving the duplicate-spawn failure mode this change is supposed to prevent.

Useful? React with 👍 / 👎.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@src/orchestrator/factory.ts`:
- Around line 1292-1293: The in-flight dispatch lock in the factory is using a
path/uuid-sensitive key via issueKey(decision.issue), which can differ from the
stable issue identity used by the dispatch-attempt blocking logic. Update the
in-flight lookup in the dispatch flow to use the same stable per-issue key as
the attempt blocker (the issue.key identity on decision.issue) so concurrent
events for the same Linear issue cannot bypass dedupe and race into duplicate
spawn handling. Keep the change localized around the code that reads
this.#dispatchInFlight and compares against the existing attempt-blocking logic.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro Plus

Run ID: 20fc9f29-03d7-48dc-b7e7-8c69ad61dedc

📥 Commits

Reviewing files that changed from the base of the PR and between 47b7b83 and 04d1441.

📒 Files selected for processing (1)
  • src/orchestrator/factory.ts

Comment thread src/orchestrator/factory.ts Outdated
@khaliqgant khaliqgant force-pushed the fix/live-dispatch-inflight-lock branch from 04d1441 to 325d2c5 Compare June 24, 2026 08:09
@agent-relay-code

Copy link
Copy Markdown
Contributor

ℹ️ pr-reviewer: review only — no file changes were applied to the PR (nothing to commit after review). The notes below are advisory and were not pushed.

Review: PR #44fix(factory): suppress concurrent duplicate dispatches

Summary

The PR wraps FactoryLoop.dispatch() with an in-memory #dispatchInFlight map keyed by issueKey(decision.issue) (key:uuid:path). A concurrent call for the same key returns the already-running promise and bumps a dispatchDuplicateSuppressed counter; the original body is moved verbatim into #dispatchUnlocked. The wrapper itself is well-formed (sets the map entry before the first await yields, and the finally only deletes its own promise).

Blocking finding — re-entrant self-deadlock (PR-introduced regression)

The new lock deadlocks on the Slack triage-escalation replay path, which re-enters dispatch() for the same issue key while the outer dispatch is still unresolved:

  • dispatch() stores promise P in #dispatchInFlight[key], then calls #dispatchUnlocked (factory.ts:1299).
  • #dispatchUnlocked takes the escalation branch and awaits #escalateTriageToSlack (factory.ts:1340) → #postAndWatchSlackEscalationThread#watchSlackThread, which replays an existing human answer inline within that await chain.
  • The replay reaches #startOrQueueSlackClarifiedDecision, which calls this.dispatch(decision, …) (factory.ts:3812). dispatchAfterSlackClarification/issueWithSlackClarification preserve the issue ref, so issueKey is identical to the outer call.
  • The inner dispatch() finds P in the map and returns it (factory.ts:1293-1297), then awaits P. P cannot resolve until #dispatchUnlocked finishes — which is blocked on this very replay. Deadlock.

Reproduced deterministically: src/orchestrator/factory.test.ts:6846 ("replays an existing human Slack answer when a triage escalation watcher starts") times out at 5000ms with the PR applied. The test is pre-existing (not in this PR's diff) and passed before the lock was added, so this is a regression that fails CI (npm test).

I did not patch this. The fix sits squarely in dispatch / in-flight lock logic, which is safety-critical and requires human judgment (the correct shape — e.g., having the re-entrant clarified-dispatch path call #dispatchUnlocked directly, or scoping the lock so the replay doesn't await its own parent promise — changes runtime behavior and must not be auto-edited). A human-authored patch is needed; the regression test at factory.test.ts:6846 already guards the correct behavior, so no test change should be required once the code is fixed.

Verification performed (CI-equivalent): npm cinpm run build (tsc + tsc-alias) passes clean; npm test (vitest) = 545 passed, 1 failed (the deadlock above).

Addressed comments

  • No bot or human review comments were present in the harness inputs (.workforce/context.json carries no review data; no comment files in .workforce/). Nothing to account for.

Advisory Notes

  • The committed package-lock.json is stale (records version 0.1.5 while package.json is 0.1.11); npm ci regenerates it with churn. Out of scope for this PR — I restored the lockfile so no unrelated change is pushed. Worth a separate lockfile-refresh PR.

Working tree

Clean — no edits left behind (the npm ci lockfile change was reverted with git restore; the untracked memory/ directory is pre-existing and unrelated).

Not printing READY: a required CI check (npm test) is failing due to the deadlock regression, and the needed fix is safety-critical dispatch/in-flight code that must be authored by a human rather than auto-applied.

@khaliqgant khaliqgant force-pushed the fix/live-dispatch-inflight-lock branch from 325d2c5 to c010b20 Compare June 24, 2026 08:12
@khaliqgant

Copy link
Copy Markdown
Member Author

Addressed PR feedback and CI failure.

Changes:

  • Included effective dry-run/live mode in the dispatch lock key so dry-runs cannot suppress real dispatches.
  • Used stable Linear issue key identity for the lock instead of the path/uuid-sensitive batch key.
  • Split escalation vs dispatch lock phases so Slack clarification replay can re-enter dispatch without self-deadlocking.
  • Added regression coverage for same Linear key through different mirror paths.

Validation:

  • npm run build
  • npm test
  • npx vitest run src/orchestrator/factory.test.ts -t "does not double-dispatch the same issue from concurrent live events"
  • npx vitest run src/orchestrator/factory.test.ts -t "dedupes concurrent dispatches for the same Linear key across mirror paths"
  • npx vitest run src/orchestrator/factory.test.ts -t "replays an existing human Slack answer when a triage escalation watcher starts"

GitHub CI package check is now passing on commit c010b20.

@agent-relay-code

Copy link
Copy Markdown
Contributor

Review of PR #44fix/live-dispatch-inflight-lock

Summary of change

The PR adds an in-process single-flight guard around FactoryLoop.dispatch. The public dispatch method now computes a per-issue key (<issue.key>:<dry-run|live>:<dispatch|escalation>), memoizes the in-flight Promise<DispatchResult> in a new #dispatchInFlight map, returns the existing promise for concurrent duplicates (incrementing dispatchDuplicateSuppressed), and delegates the real work to a renamed private #dispatchUnlocked. A test verifies two concurrent dispatches for the same Linear key arriving via mirror paths (canonical + alias) collapse to one dispatch.

Correctness & safety assessment

  • The single-flight pattern is implemented correctly: the promise is stored before await, duplicates return the cached promise, and the finally cleanup uses an identity check (this.#dispatchInFlight.get(key) === dispatched) so a newer entry under the same key is never clobbered. (factory.ts:1291)
  • #dispatchUnlocked is a pure rename of the previous dispatch body — no behavioral lines were altered inside it.
  • The new in-memory #dispatchInFlight map is distinct from the existing persistent #clearDispatchInFlight/#state.releaseInFlight mechanism (factory.ts:1858); they operate at different layers and do not conflict.
  • The dedupe key includes phase and dryRun, so a dry-run and a live dispatch, or a dispatch vs. an escalation for the same key, are not incorrectly collapsed. This preserves existing semantics.
  • No fail-closed→fail-open changes, no guard-default changes, no truthiness→presence swaps.

I did not auto-edit any of this code. It is dispatch/in-flight logic — safety-critical territory — but it is the declared purpose of this PR (an intentional, human-authored concurrency fix), and I found no defect requiring a fix. There was nothing mechanical (lint/format/typo/import-order) to clean up in the diff.

Verification (full CI command run end-to-end)

  • npm ci — ok
  • npm run build (tsc -p tsconfig.build.json && tsc-alias) — typecheck clean
  • npm test (vitest run) — 32 files, 547 tests, all passing (new test included)
  • npm pack --dry-run — ok

Addressed comments

  • No bot or reviewer comments were present in the harness materials (.workforce/context.json contains only PR metadata; no review-comment file was provided). Nothing to address.

Advisory Notes

  • None.

The change is small, well-scoped to its stated purpose, correct, and the full CI build/test/pack passes locally. No required checks appear pending and there are no findings requiring code changes; the remaining merge decision is a human judgment call.

@khaliqgant khaliqgant merged commit f05529c into main Jun 24, 2026
2 checks passed
@khaliqgant khaliqgant deleted the fix/live-dispatch-inflight-lock branch June 24, 2026 08:26
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant